Optimal Quantum Filtering and Quantum Feedback Control 
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Abstract 



Quantum mechanical systems exhibit an inher- 
ently probabilistic nature upon measurement. 
Using a quantum noise model to describe the 
stochastic evolution of the open quantum sys- 
tem and working in parallel with classical inde- 
terministic control theory, we present the the- 
ory of nonlinear optimal quantum feedback con- 
trol. The resulting quantum Bellman equation 
is then applied to the explicitly solvable quan- 
tum linear-quadratic-Gaussian (LQG) problem 
which emphasizes many similarities with the 
corresponding classical control problem. 



1 Introduction 



With technological advances now allowing the 
possibility of continuous monitoring and rapid 
manipulations of systems at the quantum level 
[UI2|, there is an increasing awareness of the ap- 
plications and importance of quantum feedback 
control. Such applications include the engineer- 
ing of quantum states, stability theory, quantum 
error correction and substantial applications in 
quantum computation H E]. This 
current interest marks quantum control theory 
as a highly rewarding branch of control the- 
ory for study and as such there is a growing 
number of recent publications on the subject 
[3 EH EH IH El IH In particular, [H] con- 



tains a useful introduction to quantum proba- 
bility and along with |15| gives a comprehensive 
discussion on the comparison of classical and 
quantum control techniques and we refer the un- 
familiar reader to these articles and references 
within. 

The main ingredients of quantum control are es- 
sentially the same as in the classical case. One 
controls the system by coupling to an exter- 
nal control field which modifies the system in 
a desirable manner. The desired objectives of 
the control can be encoded into a cost function 
along with any other stipulations or restrictions 
on the controls such that the minimization of 
this cost indicates optimality of the control pro- 
cess. There are two types of dynamical control 
- open loop (or blind) control where the controls 
are predetermined at the start of the experiment 
and closed loop (or feedback) control where con- 
trols can be chosen throughout the experiment 
and thus is preferable for stochastic dynamics. 
Previous work on the theory of optimal quantum 
open loop control includes variational techniques 
on closed qubit systems ^3 HZ) > which was also 
extended to open (dissipative) quantum systems 
|18j . However, this approach can only seek lo- 
cally optimal solutions which can often be im- 
proved further with measurement and feedback, 
since an open quantum system inevitably loses 
information to its surrounding environment. 

Quantum feedback control was formally initi- 
ated by Belavkin in a series of papers |19ll20ll?T] 
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in the 80s. This work was developed as a quan- 
tum analogy to the classical theories of nonlinear 
(Stratonovich) filtering and Bellman dynamic 
programming. In fact, the separation lemma of 
classical control theory was shown also to hold 
in the quantum domain. That is, the problem of 
optimal quantum feedback control is separated 
into quantum filtering which provides optimal 
estimates of the stochastic quantum variables 
(operators) and then an optimal control problem 
based on the output of the quantum filter. The 
quantum noise which we filter out comes from 
the disturbance to the system due to the quan- 
tum measurement. Unlike classical systems, this 
is an unavoidable feature of quantum measure- 
ment since the quantum system is not directly 
observable. The quantum filter describes a clas- 
sical stochastic process, albeit on the space of 
quantum states, so Belavkin showed how one 
can progress using tools from classical feedback 
control theory when applied to sufficient coordi- 
nates of the system [3D]. However, the lack of 
urgency for such a theory and the complexity of 
the mathematical language at the time left this 
work relatively undiscovered only to be redis- 
covered recently in the physics and engineering 
community. 

The purpose of this paper is to build on the 
original work of Belavkin and present an ac- 
cessible account of the theory of nonlinear op- 
timal quantum feedback control. Firstly we 
introduce the necessary concepts from modern 
quantum theory including quantum probability, 
non-demolition measurement, quantum stochas- 
tic calculus and quantum filtering. Next the 
quantum Bellman equation for optimal feed- 
back control with diffusive non demolition mea- 
surement is derived. Often in optimal control 
problems of this nature, the separation lemma 
is assumed and the control objectives are de- 
fined in terms of posterior sufficient coordinates 
{1511111 H2| . In this paper, we show how the gen- 
eral Bellman equation is applied with the same 
effect by application to the many dimensional 
quantum LQG problem. Next a physcial exam- 
ple of LQG control is given and we conclude with 
a discussion on the results with comparison to 



the corresponding classical control problem. 

2 Optimal quantum mea- 
surement and filtering 

This section highlights the differences between 
quantum and classical systems and introduces 
the problem of quantum measurement. After 
the appropriate setting is given, the measure- 
ment problem is then restated as a problem 
of optimal estimation of the output of a noisy 
quantum channel. Finally, the quantum filtering 
equation describing the dynamical least squares 
estimator is given. 



2.1 Quantum Probability 

Quantum physics which deals with the unavoid- 
able random nature of the microworld requires 
a new, more general, noncommutative probabil- 
ity theory than the classical one based on Kol- 
mogorov's axioms. It was developed through 
the 70s and 80s by Accardi, Belavkin, Gardiner, 
Holevo, Hudson and Parthasarthy [2~2l H3I l2"31 
amongst others. 

The essential difference between classical and 
quantum probability is that classically, Kol- 
mogorov's probability axioms allow the occur- 
rence of simultaneous events only. This is be- 
cause the classical events are described by in- 
dicator functions 1a (w) of the measurable sub- 
sets A C (] on the space of point states 17. 
They are the building blocks for the classical 
random variables described by measurable func- 
tions x : Q, — > R as linear combinations (inte- 
grals) of the indicator functions 1a- Such classi- 
cal essentially bounded variables represented by 
operators of multiplication by the corresponding 
functions, form an abelian (commutative) von 
Neumann algebra on the Hilbert space L 2 (f2,P) 
of square-integrable random functions with re- 
spect to a probability measure P. 
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In quantum probability, there are some events 
which cannot occur simultaneously, so we must 
generalize the framework of classical probabil- 
ity to incorporate these features. This is done 
by considering quantum events as self adjoint 
orthoprojectors P 2 = P = P* (where * de- 
notes the Hermitian adjoint) acting in some 
Hilbert space H not only by multiplications on 
the indicator functions 1a (oj). Quantum ran- 
dom variables are also built from events as lin- 
ear (integral) combinations of their projectors P. 
The events are incompatible if the correspond- 
ing projectors do not commute, i.e. [Pi,Pj] := 
PiPj — PjPi 7^ and therefore cannot be rep- 
resented classically by the indicator functions 
which always commute. 

One can form the non commutative von Neu- 
mann algebra A of bounded quantum random 
variables generated by the self adjoint projec- 
tors {P 1 , ..,P m }. This algebra is equal to its 
double commutant A :— {P 1 , P m }" where 
the commutant of a set S C B(H) in the algebra 
B(H) of all bounded operators on Ti is defined 
by S' := {X' e B(H) s.t. [X,X'\ = 0Vl€ S}. 
The quantum state on A, given by a positive 
operator p = p* > with unit trace Tr[p] = 1, 
defines all expectations 



(X)=T r [pX] = (p,X) 



(1) 



for operators X £ A. So we describe a quan- 
tum probability space by the pair (^4, p) . In the 
case where A is an abclian von Neumann op- 
erator algebra, there is a natural isomorphism 
(A,p) ~ L°°(Q, P) with bounded functions on 
the classical probability space (SI, P) and so we 
recover the classical statistics. 

The incompatibility of quantum events means 
that after one has observed an event, the state 
of the system needs to be updated to account for 
the change to the system or back-action affect- 
ing the expectations of all other incompatible 
events. This state change was traditionally de- 
scribed by the normalized projection postulate 



Pi 



(2) 



which also ensures instantaneous repeatability 
of the observed event corresponding to the pro- 
jection Pi. However, it has long been known 
that this phenomological description is inade- 
quate, since it fails to describe continuous mea- 
surements and experimentally it is not possible 
to perform a direct measurement of eigenstates 
of such a quantum operator. Instead we must 
consider an indirect measurement of operators 
in a coupled semi-classical field and describe the 
state change p — > pi by an optimal estimator 
based on the results of measurements in this 
field. Let T denote the Hilbert space of the 
field, which we view as a noisy measurement 
channel in the initial vacuum state (p. We only 
observe compatible events in the channel (cor- 
responding to output meter readings for exam- 
ple). So we describe these events by commuting 
projectors {P w } w eo which can be represented 
by classical indicator functions and generate the 
abelian subalgebra B C B(T) where SI is now 
the space of measurement results (eigenvalues) 
for these commuting operators. So the field op- 
erators W € B which are linear combinations of 
the commuting projectors are in one-to-one cor- 
respondence with classical random variables as 
functions w : ft — > M on the data space SI. In the 
quantum noise model, we consider input quan- 
tum noises as quantum random variables repre- 
sented by operators in the full field algebra B(T) 
of bounded operators on T which perturb the 
quantum system in such a way to allow a classi- 
cal correlated output. This interaction between 
the open quantum system and the semi-classical 
field is described on the composite system by a 
unitary operator U, which for an initial state <j> 
of the field gives the state evolution 

p^U{p®<j>)U* 

called the prior state. The reduced conditional 
evolution can then be described by the nonlinear 
map 

_ Tr F [U(p®<j>)U*(I®P„)] 
P ^ Pul Tr[U(p®<l>)U*(I®P u )] [> 

called the posterior state which is the Bayes 
law of conditioning for the measurement result 
u> G SI, normalized with respect to the output 
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probabilities P(w) = Tr[U{p®</>)U*{I®Pu)] and 
Tr^r denotes the partial trace over T . 

We denote the posterior state as a classical ran- 
dom variable p, : f2 — > .4* taking values p w in 
the space A* of states on *4. The posterior state 
gives the conditional expectation 

E[X'\Y] = (p.,X) 

which is the least squares estimator of the sys- 
tem operator X' = U*(X <£> I)U after inter- 
action, with respect to the output operators 
Y := U*(I <g> W)U. We now describe the appro- 
priate model for the dynamical coupling between 
the open quantum system and the field. 



2.2 The Quantum Vacuum Noise 
Model and Markov Approxi- 
mation 

The indirect measurement of the quantum sys- 
tem is via a coupled measurement channel, play- 
ing the role of a quantum noise bath. It is mod- 
elled by the symmetric Fock space T over the 
single particle space L 2 (R + — * Q) of square inte- 
grable functions from [0, oo) into a Hubert space 
Q of the bath degrees of freedom. Having in 
mind the vacuum noise model of the bath, let 
W := B{T) denote the quantum noise algebra 
of bounded operators on T initially in the vac- 
uum state 4>. From the divisibility property of 
the symmetric Fock space, we can factorize the 
noise algebra 

W = Wg®W t ~ T = T m ® F %oo) 

for arbitrary t > where = B{T[ a fi)) 

and F[ a .b) is the symmetric Fock space over 
L 2 {[a,b) -> Q) for < a < b. This tensor in- 
dependence implies compatibility for operators 
belonging to the disjoint time intervals of the 
noise algebra. The time evolution of the quan- 
tum system and the quantum noise bath (which 
together form a closed composite quantum sys- 
tem) can be described in the interaction repre- 
sentation by a family {Ut}tes. + of unitary opera- 



tors Ut ■ W0j 7 [o,t) — * W<8>.F[o,t)- In the weak cou- 
pling limit (short bath memory), they 
describe the Markovian flow jt : A — > A <8 Wq 
by j t (X) := U;{X ® I)U t for operators Ihi 
(we use the symbol X h A to denote that X is 
an element of A, or that its spectral projectors 
belong in A for the case of unbounded X). We 
complete the description of the joint system and 
field evolution by introducing the unitary shift 
operator St '■ -^"[o.s) ~^ •^ 7 [t,s+t) which models the 
free evolution in the field. Thus the combined 
evolution and interaction on the composite sys- 
tem is given by a family of endomorphisms {7t} 
on.4<g)Wsuch that 7t(X® W) = U*{X® W)U t 
for unitaries U t := (/ <$ S t )U t . This gives the 
cocyle identity Ut+ S = S- s UtS s U s for the inter- 
action unitaries {Ut}- Note that for ease of pre- 
sentation, we avoid the repetition of tensoring 
with the identity on Ji and and assume 
the domain of the operators is clear from the 
context. 

We now briefly discuss quantum stochastic cal- 
culus, a necessary tool when developing a time- 
continuous theory of quantum stochastic evolu- 
tion. 



2.3 Quantum Stochastic Calculus 

In this paper we consider feedback control based 
on a homodyne detection scheme. This is 
the quantum analogue of measurement of the 
Wiener process in the field and is described 
by the field quadrature Wt = A t + A$ where 
A t h Wq is called the annihilation operator on 
T . The properties of A t are such that W" t := 
cxp(i9)A t + exp(— \0)AI is equivalent to the clas- 
sical Wiener process for each 9 £ [0, 27r), how- 
ever they do not commute for different 9, so by 
considering solely the measurement of Wt, we 
restrict ourselves to a chosen classical diffusive 
measurement process corresponding to 9 = 0. 

Hudson and Parthasarathy developed 
the theory of quantum stochastic calculus us- 
ing the annihilation process and its adjoint, the 
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creation process A* t as the fundamental diffu- 
sive adapted processes and defined the interac- 
tion unitaries {Ut} as the unique solutions to the 
quantum stochastic differential equation which 
we chose of the simple form 

dUt + KU t ®dt = LU t ® dA* t -L*U t ®dA t . (4) 

with U = I. Here K = jrH + \L*L, H is the 
Hamiltonian of the quantum system and L is the 
operator describing the coupling of the system 
to the measurement channel. The increments dt, 
dAt, dAl are considered as operators acting in 
F[t,t+dt) and define stochastic Ito calculus using 
the product rule 

d(M t N t ) = d(M t )N t + M t d(N t ) + d(M t )d(Nt) 

for adapted quantum stochastic processes M t , 
N t where the quantum Ito correction term (last 
term) is calculated using the multiplication table 

(dt) 2 = 0, dtdA t = = dtdA*, 
dA* t dA t = 0, dA t dA* t =dt. ( '' 1 

2.4 Quantum Langevin Equations 
and Non-demolition Measure- 
ments 

From the quantum Ito formula applied to Xt — 
Uf(X (g> I)Ut and the quantum Ito multiplica- 
tion table jnj, we obtain the quantum Langevin 
equation 

dX t = Ct[X t ]®dt+[X u L t ]®dA*-[X u L* t ]®dA t . 

(6) 

Here C t [X t ] = j t {C[X]) is the time evolved Lind- 
blad (or Gorini-Kossakovski-Sudarshan) genera- 
tor EEHH] 

C[X] = ±[H,X} + ±(L*[X,L} + [L*,X]L) (7) 

for the semigroup of completely positive maps 
describing the dissipative evolution in the 
Markovian limit. The dual C* of this map de- 
scribes the unconditional dissipative evolution of 
states 

= ~[H, A + \{L[p\L*] + [L, p*]L*) (8) 



called the master equation which is the quantum 
analogue of the Focker-Plank equation. A time 
continuous measurement of the field quadrature 
Wt in the output channel represents an indirect 
measurement of the evolved generalized coordi- 
nate Lt + Ll h At as can be seen from the quan- 
tum Ito formula applied to the output operators 
Y t = U t *(I®Wt)U t : 

dY t = (L t + L*) ®dt + I(g>dW t . (9) 

Note that the output process Y t is directly ob- 
servable as it is a commutative family of self- 
adjoint operators {Y s } 8 <t unitary equivalent to 
the family {W s } s <i for each t. This simply fol- 
lows from the following lemma which was first 
observed by Belavkin in |19 | .|3U | . 

Lemma 1. The input and output operators sat- 
isfy the quantum non- demolition (QND) condi- 
tion 

[X t ,Y s ]=0 [Y t ,Y s ]=0 V0<s<t (10) 

Proof. Let t = s+r, r > 0, then from the cocycle 
identity we get 

u: +r (i ®w s )u s+r = 

U* s {S- s U r S s )*{I ®W s )(S- s U r S s )U s = 
U:(I®W S )U S =Y S 

where the last step uses the commutativity of 
S- s U r S s h A <g) W s + r and W s h W§. So 
[X t ,Y„] = U*[X,W s ]U t = and [Y U Y S ] = 
Ut[Wt,W s ]Ut = follows from the tensor in- 
dependence of X, W s and W t for all s ^ t. □ 



2.5 Quantum Filtering 

Classically, filtering equations are used when we 
need to estimate the value of dynamical vari- 
ables about which we have incomplete knowl- 
edge due to an indirect observation. For ex- 
ample, the Kalman-Bucy filter [2], (S3 gives a 
continuous least-squares estimator for a Gaus- 
sian classical random variable with linear dy- 
namics when we only have access to a corre- 
lated, noisy output signal. Since closed quantum 
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systems are fundamentally unobservable unless 
they are open, e.g. disturbed by quantum noise 
processes (c.f. filtering of quantum 

noise plays an important role in quantum mea- 
surement. Belavkin was the first to realize that 
an optimal estimation without further distur- 
bance is possible in the Markovian limit and is 
based on an output nondemolition measurement 
|19|.|3U|.|21). He constructed the quantum fil- 
tering equation which describes the evolution of 
the optimal estimate given by the density matrix 
conditioned on a classical output of the noisy 
quantum channel. This is used to estimate arbi- 
trary input operators X t h At which are driven 
by environmental quantum noises. The previous 
lemma shows that the expectation of X t is not 
disturbed when we measure Y s for < s < t. 
This is necessary for the existence of a well de- 
fined conditional expectation of X t with respect 
to past measurement results of Y s . 

Let C l s := {Y*}" be the abelian von Neu- 
mann algebra generated by the output opera- 
tors Y* := {Y r \s < r < t} (or their spectral 
projectors in the case of unbounded Y r ). Also 
let A\ = {X r \s < r < t}" denote the von Neu- 
mann algebra generated by the system operators 
X r h A r - From the QND condition, Cq lies in 
the center of (i.e. it is a subalgebra commut- 
ing with the whole of) Bj C A ® Wq , where 
Bj :— Aj V Cq is the smallest von Neumann 
algebra containing Af and Cq as subalgebras 
for < t < T. This gives the necessary con- 
ditions for the existence of a conditional expec- 
tation [33], defined as a linear, normcontractive 
projection Eq : Bf — > Cq. 

The conditional expectation E[X t |Yp] :— E^Xt] 
gives the least squares estimator X t of an op- 
erator X t h At conditional on the output op- 
erators Yq and so is equivalent to a classical 
random variable on the space of measurement 
trajectories il :— {w s \0 < s < t s.t. ui s is an 
eigenvalue of Y s }. This conditional expectation 
is most conveniently written in the Schrodinger 
picture E^X^ = (pl,X) for the solution 
to the classical stochastic nonlinear differential 



equation 

dpi = jC*[pi}dt + a(pi){dY t -{pl,L + L*}dt) 

(11) 

often called the Belavkin quantum filtering 
equation, where 

<T(p t .) = p t .L* + Lp t .-(pi > L* + L)pi 

is the nonlinear fluctuation coefficient. 

We can generalize the filtering equation to the 
case where we couple the open quantum sys- 
tem to d independent measurement channels. If 
we assume no scattering between the channels, 
then the family of unitary operators {Ut}tem + 
describing the evolution in the interaction pic- 
ture U t :H® T® d t) ^H® T® d t) satisfy 

d 

dU t + KU t ®dt = YyUUt^dA*^- L*U t ®dA^ t ] 

z=l 

where Li describes the coupling to the «th chan- 
nel and K = jrH + \ Yfi=x L * L i- Through- 
out this paper we reserve the Roman charac- 
ter i to denote the imaginary unit i := \J— I, 
whereas italic i is freely used as an index. Note 
that we have tensor independence of the anni- 
hilation increments dAij, dAjj for i ^ j, so 
the quantum vacuum noises commute for dif- 
ferent channels. The Belavkin filtering equa- 
tion for a simultaneous diffusive measurement 
of Y i>t =U*(I ®W iit )Ut gives 

d 

dp 1 . = C^dt+^aiipDidY^-iplLi+LDdt) 

i=l 

(12) 

for W it t = (A iit + Al t ). 

3 Optimal Quantum Con- 
trol 

We now couple the system to a control field. If 
we assume no scattering between the measure- 
ment and control fields and assume a weak cou- 
pling such that information is not lost into the 
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control field, then this effectively replaces the 
Hamiltonian H of the system with a controlled 
Hamiltonian H(u s ) for admissible real valued 
control functions u s £ K say, at time s. This 
Hamiltonian generates the controlled unitaries 
Ut(u ) giving the controlled flow 

jt{ul)[X] :=[/;(4)(X®/)f/ t (4) 

where u Q :— {u s \0 < s < t} is the control 
process over the interval [0,t). The controlled 
posterior density operator p\{uq) can then be 
obtained from with the controlled Hamil- 
tonian H(u t ) which appears in the controlled 
Lindblad term C{u t ). 

In classical control, we can allow complete ob- 
servability of the controllable system, so that 
feedback controls are determined by the sys- 
tem variables x t — > Ut{xt). However, in quan- 
tum systems, we do not have the point states Xt 
due to joint non observability of the system op- 
erators X t , so the stochastic feedback controls 
should be given by a function of the stochas- 
tic output process Yq which is associated with 
a classical random variable Mt(-) on fl . I.e. the 
measurement trajectory is fed into the control 
Wq h- > Mt(wg). Thus the feedback controlled flow 
is a map jt(uo(Y^)) from A to At V Cq. 

The optimality of control is judged by the ex- 
pected cost associated to the admissible control 
process u q for the finite duration T of the exper- 
iment. Admissible control strategies are defined 
as those Uq for which the operator valued cost 
integral 



j a ( U s )[C(u s )]ds+j T ( U T)[S] (13) 



exists in the strong operator topology for self 
adjoint positive operators C(u s ),S h A giving 
the expected cost by the expectation 



{p®<f>,J(vfi)). 



(14) 



An optimal feedback control strategy Uq*(-) for 
nondemolition measurements of the output op- 
erators Yq is one which minimizes the expected 



posterior cost-to-go 
(P^,^0*(-))) = 



<(-)6£/ T (-) 



{p®(t>, J«(-))> 



where Uq(-) is the space of admissible stochas- 
tic control strategies Uq(-). This dynamical op- 
timization problem is considerably simplified by 
the following Lemma first observed by Bellman. 

Lemma 2 (Principle of Optimality). // 

u o*(') * s an °pt'i'' ma i strategy for the cost func- 
tion | Ity) given the initial state p ® <p, then its 
restriction uf*(-) to the interval [t,T) is optimal 
for the cost-to-go 

J t (uT(-))=^js(u s t (-))[C(u s (-))}d S . , 
+JT(uf(-))[S] 

given the state pi(u (-)) at time t. 

We can now reduce the dynamics to the observ- 
able output algebra and rewrite the expectation 
as a conditional one 

(p®<j>,J t (uJ(Y*))) = {4>lE%[Jt(£(Yg))]) 

where Eq : Bf — ► Cq is the conditional expecta- 
tion on Bj = Af V Cq which defines the feed- 
back controlled posterior density operator by 
(p* («&(■)), X) = Ei o jt{ul{-))[X] at time t. 

Theorem 1. The posterior cost-to-go from 
state p at time t satisfies 

E t [Jt(u?(-))]=E [J(t,u?(-),p)} (16) 

where 

J{t,uJ{-),p)= jf \pl{ut{-)),C{u s {-)))ds 
+{p T {uT{.)),S) 

is a random variable on Qq and p s m (uf(-)) is the 
solution to the controlled filtering equation for 
s > t with p — pi(u (-)). 

Proof. The 'quantum' conditional expectation 
Eq acting on future operators gives 

Elo 3s (ut(.))[X]=El[(pl(ut(-)),X^ 

CI 



for X h A, where Eq : Cq — > Cq is the 'classi- 
cal' conditional expectation on Cq satisfying the 
tower property E* o Eg = E for t<s<T. □ 
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Let us denote the minimum posterior cost-to-go 



S(t,p):= min E 4 [J(i, p)]. (17) 

«f(-)e(7 t r (-) 

Theorem 2. TTie minimum posterior cost-to-go 
satisfies the Bellman equation 

d 1 rf 

-g t S{t, p) + ~ Y^(tTi<J>) <8> (Ti(p), (S ® £)S(t, p)) 
1=1 



+ min{</>, <?(«,(■)) +-C(u t (-))[«(t,p)]» = 
*»«(•) 

(18) 

where SS(t, p) \- A denotes the derivation of 
S(t, p) with respect to p and o~i (p) is i/ie non- 
linear fluctuation coefficient in the filtering equa- 
tion 171)) . 



Proof. From the definition of S(i, p) and 
J(t, uf(-), p), we have 



S(*,p 4 



min En 

«?(•) 



/, <p:(«?(-)).c(u.(-)))& 

+ J(i + 6, U f +e (.),p t+e ) 



So when e —> dt becomes sufficiently small, we 
approximate this by 

S(i,p)-mmE | +5{t + dt ^t +dt) j (19) 

where we use the tower property of the classical 
conditional expectation. Assuming that S(i, p ) 
is sufficiently differentiable, we use the Taylor 
expansion 

S(t + dt, p t+dt ) = 
S(i, p 4 ) + &S(t, p*)A + (dp 4 , <5S(t, p 4 )) + 
I EtiMP*) ® '.V). (S ® <5)S(t, p 4 ))dt 

where <5S(t, p) := j-S(t,p) denotes the deriva- 
tion of S(i, p) with respect to p. Using this ex- 
pansion in l|19fl gives the Bellman equation l|18|) 
when we observe that S(t, p) + -§iS(t, p) does not 
depend on u t and E^dY^t] = for the innova- 
tion process dYi t t = dYi t t — (p 4 , Li + L*)dt. □ 



4 Application of Results to 
a Linear Quantum Dy- 



namical System 



We illustrate the ideas of quantum filtering and 
control described above by application to the 
multidimensional quantum LQG control prob- 
lem. LQG control is well studied in classical 
control theory and we shall see many similari- 
ties between quantum and classical LQG control 
theory. 



4.1 Quantum Filtering of Linear, 
Gaussian Dynamics 

Let X be the phase space vector of self adjoint 
operators X 1 , i = 1, ...,m satisfying the canoni- 
cal commutation relations (CCRs) 

[x\x j ] = x i x j - x J x i = \hJ ij i 

for i,j — 1, ...to where / is the identity operator 
on H. The CCRs can be written in vector form 
as 

[X, X T ] := XX T - (XX T ) T = ifiJJ 

where X T = (X 1 ,.-;^" 1 ) is the row vector 
transpose of X and J = (J 1 - 7 ) is an anti- 
symmetric real valued matrix which is assumed 
to be nondegenerate for an even to = 2d say. We 
couple the open quantum system to d measure- 
ment channels via the operator vector L = AX, 
where A is a d x to matrix of complex-valued 
coefficients. Let us place it in a controllable po- 
tential which is described by the Hamiltonian 

H (u t ) = 7^ Tr ^ + X T Ku t + ujK^X (20) 

for real vector valued control parameters u t € 
M. d , where R is a real symmetric to x to matrix 
and K is a complex to x d matrix. We shall use 



A* to denote complex conjugation (A*)y = 
and A^ = (A*) T the Hermitian conjugate. 



A* 



These definitions allow us to calculate the com- 
ponents of the controlled Lindblad generator 
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from with the controlled Hamiltonian 12U|) 
which we write here in vector form 

C(u t )[X] = J(R + fi3(A t A))X + J(K + K*)w t 

omitting the identity / for notational conve- 
nience where 2iSy(A^A) = A^A - A T A*. So 
from and we obtain the following quan- 
tum linear Langevin vector equation 

dX t = {AX t + Bu t )dt + dV t (21) 

and linear output equation 

dY t = CX t dt + dW t (22) 

where A := J(R + 7i3(AtA)), B := J(K + K*), 
C := A + A*. The quantum noise increments 
are given by vectors 

dV t = ihJ(A T dA* t ~ AUA t ) 
dW t = dA t + dA* t 

for (At)i — Ai.t the annihilation operator on the 
ith coupled independent measurement channel. 

Let us denote the initial mean X of the phase 
space operator vector by the component wise ex- 
pectation (Xy = X' L = (p, X 1 ) and symmetric 
covariance 

S lJ := ^(p,X l X j + X ] X l ) - X l X J . 

which is given by a real positive definite matrix 
X = (S* J ') satisfying the Heisenberg uncertainty 
principle 

S > ±p (23) 

The filtering equation (|12|l preserves the Gaus- 
sian nature of the posterior state [35], so the 
posterior mean (X t y = X\ — (p* , X 1 ) and sym- 
metric error covariances 

:=l{plX i Xi+XlX i )-X i t xi 

form a set of sufficient coordinates for the quan- 
tum LQG system and agree with the initial 
mean and covariance for p® = p. Using (|12|l . 



the posterior expectation of X t for non demoli- 
tion measurement of the output operators Y t is 
given in vector form 

dX t = (AX t + Bu t )dt + K t dY t (24) 
K t = S t C T +M (25) 

where dY t = dY t — CX t dt is the innovating 
martingale which describes the information gain 
from measurement of the output vector operator 
Yt- 

The symmetric error covariance S t satisfies the 
matrix Ricatti equation 

|S t = AS t + E t A T + N 

-(£ t C T + M)(£ t C T + M) T 
So- S 

(26) 

where 

N = -ft 2 J(A f A + A T A*)J T 

is the intensity (symmetric covariance) matrix 
of the quantum noise increment dV t and 

M = -ftJ(A T - At) 

2 v ' 

is the covariance matrix of the noise increments 
dV t and dW t . 

4.2 Quantum LQG Control 

We aim to control the phase space operator 
whilst constraining the amplitude of the con- 
trolling force for energy considerations. Thus, 
our control objectives and restraints can be de- 
scribed by the operator valued risk (jT3|) with 
quadratic parameters 

C(u s ) = X T FX + X T G T u s + u]GX + u]u s 

and S = X T QX for positive real symmetric 
to x to matrices J7,F and a real d x to matrix 
G. 

Since X and S form a set of sufficient coordi- 
nates, they describe the full probability distribu- 
tion given by p, so we may consider the deriva- 
tion of 5(t, p) as partial derivatives of S(t, X, £). 
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So from l|21|) and the Gaussian nature of the sys- 
tem, we obtain 

(p,C(u t )[6S(t,X,-E)}) = 

i(AX + Bu t ) T V ± S + ±V^S T (AX + Bu t ) 

+ (ae + sa t + n,v e s) 

£?=ifo(p) ® (<* ® *)S(t, X, S)) = 
((£C T + M)(SC T + M) T , V^S - 2V s s) 

where (D,E) := Tr[D T E] is the Hilbert- 
Schmidt inner product on the vector space of 
complex- valued m x m matrices. We denote the 
partial derivatives by (Vj^-S)j = ^jpS(t, -X",E), 

(V s S)y = Q§-S(t,X,-£) and (V^S), 3 - = 

d S S(t,X,Yi). Inserting into the Bell- 
ax ax 3 ' ... 
man equation Q18f) and minimizing gives u t = 

-(±B T V^S + GX) where S(t,X,H) now sat- 
isfies the nonlinear partial differential equation 

-fS(t,X,£) = 

i(X T A T V^S + V^S T AX) + X T FX 
+ (AS + SA T + N, V S S) + (S, F) 
-(±B T V^S + GX) T (±B T V ± S + GX) 
+ ((SC T + M)(SC T + M) T , iV^S - V S S 

(27)' 

which is called the Hamilton- Jacobi-Bellman 
(HJB) equation for this example. 

It is well known from classical control theory 
that LQG control gives a posterior cost-to-go 
which is quadratic in the posterior mean. So we 
use the ansatz 

s(t, x, s) = x T n t x + (n t , s) + a t 

in the HJB equation (|27|) . This gives the optimal 
feedback control strategy 

u t = -t t X t (28) 
L t = B T n t + G (29) 

which is linear in the solution to the filtering 
equation X t at time t where fit satisfies the 



backwards matrix Ricatti equation 

-^n*= n t A + A T a + F 

— (B T S~2 t + G) T (B T fi t + G) 

n T = n 

(30) 

and at satisfies 

-|a t = ((B T n + G) T (B T n + G),S i ) 

+ (a,N) 

O.T = 0. 

(31) 

From this we obtain the total minimal cost 
S(0,X,E) = 

x T n x + Tr[ra £] + / T Tr[rj t N]dt 

+ J T Tr[(B T tt + G) T (B T J7 + G)H t ]dt 

(32) 

where r2 is the solution to l|3U[l at time t = 0. 



4.3 Duality 

The example of the quantum LQG control prob- 
lem is important since it is one of the few exactly 
solvable control problems and emphasizes the 
similarities between the two components of op- 
timal quantum feedback control, namely quan- 
tum filtering and optimal control. The duality 
between the solutions of filtering l|24|l - (|26J) an d 
control l|28|) - (|3U[l is summarized in the duality 
table 



Filtering 


s t 




A 


C 


N 


M 


Control 


Or-t 


L'T-t 


A T 


B T 


F 


G T 



(33) 

which allows us to formulate and solve the dual 
control problem given the filtering parameters. 
The duality can be understood when we examine 
the nature of each of the methods used. Both 
methods involve the minimization of a quadratic 
function for linear, Gaussian systems, (i.e. the 
least squares error for filtering and the quadratic 
cost for control). The time reversal in the dual 
picture is explained by the forward (backward) 
induction used in the dynamical minimization 
problem for the filtering (control) problem. 
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4.4 Optimal feedback control of 
continuously observed quan- 
tum free particle 

We give a more physical interpretation of the 
above results by application to an explicit ex- 
ample of LQG control where the duality between 
filtering and control is preserved. The example 
of the complex Gaussian oscillator was given in 
|36| . however we may now use the multidimen- 
sional quantum LQG control solutions derived 
above for application on higher dimensional sys- 
tems which do not have such complex represen- 
tation. The optimal control of a continuously 
observed quantum free particle with quadratic 
cost is the simplest such example. 

Let X T = (Q,P) be the phase space vector op- 
erator consisting of the position Q and momen- 
tum P operators of the free particle having the 
initial expectations Q and P respectively. Let us 
also denote the initial dispersions by oq and op 
respectively and the initial covariance of Q and 
P by (jqp = opQ. We can perform a continu- 
ous observation of the particle by coupling the 
position operator to the measurement channel 
L — Q in which we measure the classical Wiener 
process W t — A t + A$ and the particle is con- 
trolled using the linear potential V(u t ) = —u t Q 
for Ut € K. The Hamiltonian of this simple 
system is then given by H{u t ) — ^ P 2 — u t Q 
where M is the mass of the particle and the cor- 
responding Langevin equations are 

5* = ^ Jt p * = u * + V * (34) 

where Vt is the time derivative of the Wiener 
process V t — HW^ 2 and represents the system 
process noise due to the interaction with the cou- 
pled noise bath. The operators Y t satisfy the 
linear output equation 



dt 



Y t = 2Q t + W t 



(35) 



which is perturbed by measurement noises rep- 
resented by the time derivative of the Wiener 
process Wt- 



The optimal estimates of the position and mo- 
mentum based on a non demolition measure- 
ment of Y t are then given by the quantum 
Kalman Bucy filter {21)1 



1 

M 



P t dt + 2a Q ,tdY t (36) 



dP t = u t dt + 2<jQp t dYt (37) 

where the innovation process Yj describes the 
gain of information due to measurement of Y t 
given by 

Y t = Y t - 2Q t - 

In practice, for a continuous observation, it is 
the measurement current I t := dY t /dt which we 
observe and so we write the filtering equations 
in the form 



dt 
d_ 

dt 



1 

If 



Pt+2a Qt t(It-2Qt) (38) 



Pt = ut + 2a QP ,t(It-2Qt) (39) 



where the error covariances satisfy the Ricatti 
equations 

m a Q,t = -q a QP,t-^{vQ,t) 2 

diVQP-t = M a P,t - 4<rQ,t<rQP,t (40) 

f t a P .t = h 2 - 4(a QP , t ) 2 

with initial conditions 

OQ,0 = 0Q, CTQPfi — aQp, Ppfl=<Jp. 

The Ricatti equations for the error covariance in 
the filtered free particle dynamics have an exact 
solution |37| . however we will simply comment 
on the stationary solutions which are the solu- 
tions obtained by setting the LHS of (|40|l to zero, 
giving the asymptotic behaviour of the posterior 
dispersions for t — > oo 



0p,t 



&PQ,t 



2' 

(41) 

This proper treatment dispels the paradoxical 
quantum Zeno effect which insists that a quan- 
tum state is frozen in time by a continuous ob- 
servation. Instead we can describe the contin- 
uous observation as an optimal estimation with 
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posterior dispersions tending to a finite limit sat- 
isfying the Heisenberg uncertainty relation 

A Q , t A P ,t = ^oq.top.t -> n/\/2 > 7i/2. 

In contrast, for the case without conditioning 
(where the measurement results are ignored or 
averaged over) the Ricatti equations for the 
dispersions become linear which have solutions 
tending to infinity like t 3 . This is faster than 
the t 2 spreading of the wavefunction due to 
the closed evolution described by Schrodinger's 
equation as one would expect since the coupled 
noise bath only serves to increase the dispersion. 

The dual optimal control problem can be found 
by identifying the corresponding dual matrices 
from the table ll-S-SI) which give the quadratic con- 
trol parameters 

C(u t ) = (3Q 2 + u 2 

S = ujqQ 2 +uj Q p{PQ + QP)+ujpP 2 

which for the linear Gaussian system (|34(1 gives 
the optimal control strategy 

ut = -2{uj PQtt P t + Lo Ptt Qt) (42) 

where the coefficients are the solutions to the 
Ricatti equations 

-ffPt = jgVQP* ~ 4 (wp,t) 2 
-£tjQP,t = JfWQ.t ~ 4up,tUQP,t (43) 
-|wQ,t = /3-4{u QPtt ) 2 

with terminal solutions 

^P,0 = ^P, Wqp,o = Uqp, LOQfl = LUq. 

Note that in this example, as well as identify- 
ing the dual matrices by transposition and time 
reversal according to the duality table (|33(1 , one 
must also interchange the coordinates P <-> Q. 
This is because the matrix of coefficients A is 
non-symmetric and nilpotent, so it is dual to its 
transpose only when we interchange the coor- 
dinates in the dual picture. Thus the optimal 
coefficients {u}p,uWQP,t>MQ,t} m the quadratic 
cost-to-go correspond to the minimal error co- 
variances {(TQ ! T-t,o'QP,T-t,0p,T-t} m the dual 
picture. 



The minimal total cost for the experiment can 
be obtained from (|32[l by substitution of these 
solutions 

S = LU Qfi (Q 2 + a Q ) + 2uj QPfl (QP + (jqp) 
+uj Pfl {P 2 +<rp) + J Q (h 2 ujp^ t +uj 2 PQ t a Qy t)dt 
+ J Q (u?p t o PA +2ujQp it u>p it crpQ.t)dt 

(44) 



5 Discussion 



We have shown that the optimal quantum feed- 
back control problem reduces to an optimal es- 
timation problem followed by an optimal con- 
trol problem based on this optimal estimator. 
The optimal (least-squares) estimator for quan- 
tum random variables (operators) given a classi- 
cal nondemolition output measurement process 
is the conditional expectation which is given by 
the result of the filtering equation H12JI . The re- 
sulting optimal control problem is then defined 
on the output of this filter, which reduces to a 
classical control problem on the space of quan- 
tum states. For cost functions that are linear in 
the state, the optimal feedback control strategy 
is given by the solution to the Bellman equation 

In the LQG example, the space of quantum 
states are restricted to the class of Gaussian 
states so the probability distribution is parame- 
terized by the mean and covariance of the gen- 
erating operators. However, due to non commu- 
tativity of these quantum operators there are 
many different definitions of the covariance ma- 
trices. For direct comparison to classical LQG 
control theory, we choose the symmetric repre- 
sentation of the covariance matrices, although 
unlike the classical case, the Heisenberg un- 
certainty principle 1)23(1 places a positive lower 
bound on the covariances. In particular, this 
prohibits the common classical assumption of 
uncorrelated process and measurement noise if 
the coupling to the noise bath is complex. 
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